1. Field of the Invention
The present inventions relate to video monitoring systems, and more specifically, to automatically expanding the zoom capability of a wide-angle video camera.
2. Background
Real-time video surveillance systems have become increasingly popular in security monitoring applications. A new class of cameras replaces the mechanical Pan-Tilt-Zoom (PTZ) functions with a wide-angle optical system and image processing, as discussed in U.S. patent application Ser. No. 10/837,019 entitled “Method of Simultaneously Displaying Multiple Views for Video Surveillance,” which is hereby incorporated by reference. This class of cameras is further discussed in U.S. patent application Ser. No. 10/837,325 entitled “Multiple View Processing in Wide-Angle Video Camera,” which is hereby incorporated by reference. This type of camera monitors a wide field of view and selects regions from it to transmit to a base station; in this way it emulates the behavior of a mechanical PTZ camera. The wide-angle optics introduces distortion into the captured image, and processing algorithms are used to correct the distortion and convert it to a view that has the same perspective as a mechanical PTZ camera.
The U.S. patent application Ser. No. 10/837,326 entitled, “Multiple Object Processing in Wide-Angle Video Camera” by Yavuz Ahiska, which is hereby incorporated by reference, describes a way to correct the distorted view captured by a wide-angle camera. This camera, even using this distortion-correction process, only has limited capabilities to zoom into a region of interest. The camera can also be a programmable one as described in U.S. patent application Ser. No. 10/837,325, entitled “Multiple View Processing in Wide-Angle Video Camera,” containing programmable embedded microprocessors.
There exists a conflict between a video camera's field of view and the effective resolution of its image. Wide-angle lenses rarely offer any significant optical zoom, and similarly, video cameras with a high zoom capability have restricted fields of view (especially when their magnification is increased).
A solution to monitoring a wide-angle area while being able to capture regions at a higher detail is to utilize multiple cameras at differing locations. The U.S. Pat. No. 6,724,421, which is hereby incorporated by reference, and the public domain document, “A Master-Slave System to Acquire Biometric Imagery of Humans at Distance,” by X. Zhou et al, which is hereby incorporated by reference, describe systems using multiple cameras to monitor a wide-angle area. In these systems, a separate base station unit controls the two cameras monitoring the scene. In addition, these systems do not try to expand the zoom function of the master camera.
The U.S. Pat. No. 6,147,709, which is hereby incorporated by reference, describes a method and apparatus for overlaying a high-resolution image onto a hemispherical interactive image captured by a camera by matching at least three points between the high-resolution image and the perspective corrected image. A major drawback with this process is that it makes comparisons in the perspective corrected domain.
Moving regions in a video corresponding to persons or moving objects, together with tracked objects which may no longer be moving, and their local neighborhoods in the video define Regions of Interest (RoI) because persons, moving and/or tracked objects, etc. are important in security monitoring applications. In order to provide real-time alarms for dangerous events, RoI should be tracked and zoomed for closer inspection. Conventional Closed Circuit Television (CCTV) systems, which only capture recorded video for later analysis, cannot provide automatic alarm and event triggers without delay.
A wide field of view camera that can both monitor a wide-angle scene, while also being able to simultaneously and automatically capture regions of interest at a greater magnification is very desirable in surveillance systems. For example, a high-resolution image could make the difference in positively identifying a criminal committing an offense or the detail surrounding an unattended suitcase. Therefore, it is very important to provide a high-resolution view of a person in a surveillance application.
Wide-angle surveillance is necessary in many CCTV applications. Cameras such as dome cameras and cameras with fisheye or peripheral lenses can produce wide-angle video. A major weakness of wide-angle surveillance cameras and systems is that they either do not have the capability to zoom into a RoI or are limited in their zooming capability.
The system can also have a computer program comprising a machine-readable medium having computer executable program instructions thereon for executing the moving object detection and object tracking algorithms fully in the programmable camera device as described in U.S. patent application Ser. No. 10/924,279, entitled “Tracking Moving Objects in Video Using Wavelet Domain Information,” by A. E. Cetin and Y. Ahiska, which is hereby incorporated by reference. Automatic moving-object detection and object tracking capability of the wide field of view camera can define a RoI in the wide-angle scene monitored by the camera containing the object in question. As this RoI will be of interest in many security applications, the region can be tracked by the electronic PTZ capability of the master camera.
There is a present demand for a system that can both monitor a wide area, while also being able to simultaneously and automatically capture regions of interest at a higher resolution.
Automatically Expanding the Zoom Capability of a Wide-Angle Video Camera
The present innovations include a new approach that achieves the ability to monitor a wide-angle area while being able to capture regions of higher detail.
In one example embodiment, a wide-angle, master camera, such as a dome camera or a camera with a fish-eye or peripheral lens, preferably with substantially no zoom capabilities, is used to capture images and automatically identify RoI, e.g. motion detecting and/or object tracking. In this embodiment, at least one other camera, preferably with expanded zoom capabilities relative to the master camera, can be used to zoom into the identified RoI. The views from the cameras other than the master camera can be used for several purposes including, but not limited to, input into the master camera or output to a base station.
In another example embodiment, control circuitry sends PTZ controls to one or more slave cameras based in at least partial dependence on the wide-angle images captured by the master camera. Among other things, these controls can be used to aim the slave camera towards the RoI and/or zoom the slave camera onto the RoI.
In another class of embodiments, the output of a slave camera is compared to the images captured by the master camera and PTZ controls are sent to one or more slave cameras based in at least partial dependence on the comparison. Output images from the slave cameras can then be used for several purposes including, but not limited to, comparing them to RoI from the master camera, outputting them to a base station, or overlaying them onto other images.
In a sample of this embodiment, after the slave camera has moved in accordance with the PTZ controls, the output from the slave camera can be compared to the images from the master camera to generate a new set of PTZ controls. This process can be, but does not have to be, used to match the output images from the slave camera to the RoI identified in the output images from the master camera. This process can be, but does not have to be, an iterative process that can be repeated to yield any level of desired matching accuracy. There are multiple methods for implementing this synchronization including, but not limited to image-processing techniques to match views, calibration procedures, or position analysis of feedback from the slave camera.
In another example embodiment, the images from the slave camera can be used to replace, correct, inset, or overlay some or all of the images from the master camera. The composite images can be used for several purposes including, but not limited to, recording them, outputting them to a base station, and/or using them to generate PTZ controls.
In another embodiment, several slave cameras, preferable monitoring different regions, can be used and the perspective-corrected view of the master camera can be altered in at least partial dependence on the adjusted views of at least one of these slave cameras.
In another embodiment, motion-detecting software can be utilized to define the RoI as moving regions in the video corresponding to, but not limited to, persons, moving objects, tracked objects which may no longer be moving, and/or their local neighborhoods in the video.
These and other embodiments of the present innovations are described more fully below.